Soft Margin Consistency Based Scalable Multi-View Maximum Entropy Discrimination
نویسندگان
چکیده
Multi-view learning receives increasing interest in recent years to analyze complex data. Lately, multiview maximum entropy discrimination (MVMED) and alternative MVMED (AMVMED) were proposed as extensions of maximum entropy discrimination (MED) to the multi-view learning setting, which use the hard margin consistency principle that enforces two view margins to be the same. In this paper, we propose soft margin consistency based multi-view MED (SMVMED) achieving margin consistency in a less strict way, which minimizes the relative entropy between the posteriors of two view margins. With a trade-off parameter balancing large margin and margin consistency, SMVMED is more flexible. We also propose a sequential minimal optimization (SMO) algorithm to efficiently train SMVMED and make it scalable to large datasets. We evaluate the performance of SMVMED on multiple real-world datasets and get encouraging results.
منابع مشابه
Multi-View Maximum Entropy Discrimination
Maximum entropy discrimination (MED) is a general framework for discriminative estimation based on the well known maximum entropy principle, which embodies the Bayesian integration of prior information with large margin constraints on observations. It is a successful combination of maximum entropy learning and maximum margin learning, and can subsume support vector machines (SVMs) as a special ...
متن کاملMulti-kernel maximum entropy discrimination for multi-view learning
Maximum entropy discrimination (MED) is a general framework for discriminative estimation which integrates the principles of maximum entropy and maximum margin. In this paper, we propose a novel approach named multi-kernel MED (MKMED) for multi-view learning (MVL), which takes advantage of the complementary principle for MVL. Multiple kernels encode the similarities in different views. We obtai...
متن کاملMistake Bounds for Maximum Entropy Discrimination
We establish a mistake bound for an ensemble method for classification based on maximizing the entropy of voting weights subject to margin constraints. The bound is the same as a general bound proved for the Weighted Majority Algorithm, and similar to bounds for other variants of Winnow. We prove a more refined bound that leads to a nearly optimal algorithm for learning disjunctions, again, bas...
متن کاملMaximum Entropy Discrimination Markov Networks
Standard maximum margin structured prediction methods lack a straightforward probabilistic interpretation of the learning scheme and the prediction rule. Therefore its unique advantages such as dual sparseness and kernel tricks cannot be easily conjoined with the merits of a probabilistic model such as Bayesian regularization, model averaging, and ability to model hidden variables. In this pape...
متن کاملGini Support Vector Machine: Quadratic Entropy Based Robust Multi-Class Probability Regression
Many classification tasks require estimation of output class probabilities for use as confidence scores or for inference integrated with other models. Probability estimates derived from large margin classifiers such as support vector machines (SVMs) are often unreliable. We extend SVM large margin classification to GiniSVM maximum entropy multi-class probability regression. GiniSVM combines a q...
متن کامل